1. Field
Apparatuses and methods consistent with example embodiments relate relates to an image processing system and a method for detecting light sources in a multi-illuminated environment using a composite red-green-blue-infrared (RGB-IR) sensor.
2. Description of Related Art
The color of objects viewed in a scene is affected by the lighting conditions under which the scene is viewed. A viewer's eyes and brain compensate for different types of light. White object appears white to a viewer whether it is viewed in sunlight, or indoors under incandescent or fluorescent light. In cameras, an auto-white balance (AWB) function helps to compensate for different types of lighting, to render a white object white. Therefore, correct illuminant detection (light source detection) is an important part of accurately capturing any scene.
Illuminant estimation is a primary step during image processing in a camera that uses an AWB process to remove a color cast in an image and thus improve the color in the scene. Existing AWB processes uses only single light source or illuminant. Therefore, it is difficult to detect multiple illuminants using only a single red-green-blue (RGB) image, and thus using only a single RGB image provides incorrect detection of illuminant results in color cast. Existing methods also provide improved estimates of an illuminant only at the expense of processing time.
In the related art, a statistic-based method determines a single illuminant scenario using visible and near-infrared (NIR) spectrums. Data for the results are generated from a modified digital single lens reflex (DSLR) camera having a Bayer sensor with a NIR-cutoff filter removed manually from the sensor module. Camera can capture either visible or NIR light by placing an appropriate filter in front of the lens. The sensor does not have a specific IR pixel which can capture wavelength 850 nm+/−150 nm. The system captures visible wavelengths (390 nm˜700 nm) with an IR-cutoff-filter. The system captures from visible to NIR spectrums (wavelength 390 nm to 1200 nm) when the NIR cut-off filter is removed manually. In the absence of an NIR cut-off filter, Bayer pixels receive some amount of NIR crosstalk along with the visible spectrum which is used for an illuminant estimation. Thus, this method faces challenges in multi-illuminant scenarios.
Another related art system proposes a learning-based technique using a convolutional neural network (CNN) for single and multiple illuminant scenarios. The system and the method utilize existing datasets from a Bayer sensor and use relighting techniques to synthetically generate multi-illuminant scenarios. The results are restricted two illuminant scenarios.
Another related art proposes local estimation method a Bayer sensor to improve detection in a multi-illuminant scenario.
Based on the related art, it is observed that the required sensor pixel characteristics are not explicitly addressed, which is essential for designing an accurate illuminant estimator. The applicability of illuminant detection methods for image sensors with visible and NIR pixels or in a dual sensor with separate visible and NIR pixel array or in single visible (e.g. RGB) sensor without NIR cut-off filter have not been exhaustively studied. Further, there is needed an illuminant detection system and method for a composite RGB-IR sensor without an IR-cut-off filter.
The above information is presented as background information only to assist with understanding of the disclosure. No determination has been made, and no assertion is made, as to whether any of the above might be applicable as prior art with regard to the disclosure.